Timestamping packets in a network

ABSTRACT

Techniques are presented herein to facilitate latency measurements in a networking environment. A first network device receives a packet for transport within a network domain that comprises a plurality of network devices. The plurality of network devices have a common time reference, that is, they are time synchronized. The first network device generates timestamp information indicating time of arrival of the packet at the first network device. The first network device inserts into the packet a tag that comprises at least a first subfield and a second subfield. The first subfield comprising a type indicator to signify to other network devices in the network domain that the tag includes timestamp information, and the second subfield includes the timestamp information. The first network device sends the packet from to into the network domain to another network device. Other network devices which receive that packet can make latency measurements.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/708,347, filed Dec. 7, 2012, entitled “Timestamping Packets in aNetwork,” which claims priority to U.S. Provisional Patent ApplicationNo. 61/702,323, filed Sep. 18, 2012, also entitled “Timestamping Packetsin a Network.” Both applications are incorporated herein by reference intheir entirety.

TECHNICAL FIELD

The present disclosure relates to networking systems and devices.

BACKGROUND

In a computer network, such as a data center, data is transmitted from asource to a destination in the form of packets that generally passthrough one or more networking devices (e.g., switches, routers,firewalls, etc.). During the transmission, packets are generallytemporarily stored in one or more network buffers of the networkingdevices.

Certain data center customers demand network architectures that canprovide low latency, high bandwidth, and often massive scalability.However, measuring latency may be difficult and time sensitiveapplications often do not have the proper visibility into how it hastaken for packets to reach a certain destination and when packets wereactually sourced at specific locations in the network.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example network block diagram in which thetimestamping of packets is used to track latency at different points ina network domain.

FIG. 2 illustrates in more detail components in each network device inthe network domain that enable the timestamping and latency measurementsof packets that traverse the network domain.

FIGS. 3A-3E illustrate examples of a timestamp tag (TTAG) that may beinserted into a packet by any of the network devices in the networkdomain, and which is used for latency measurements.

FIG. 4 illustrates an example of a packet having one or more TTAGs.

FIG. 5 is a flow chart that depicts operations performed by networkdevices that insert and operate on TTAGs.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

Techniques are presented herein to facilitate latency measurements in anetworking environment. A first network device receives a packet fortransport within a network domain that comprises a plurality of networkdevices. The plurality of network devices have a common time reference,that is, they are time synchronized. The first network device generatestimestamp information indicating time of arrival of the packet at thefirst network device. The first network device inserts into the packet atag that comprises at least a first subfield and a second subfield. Thefirst subfield comprises a type indicator to signify to other networkdevices in the network domain that the tag includes timestampinformation, and the second subfield includes the timestamp information.The first network device sends the packet into the network domain toanother network device. Other network devices in the network domainwhich receive that packet can then make latency measurements, insertanother tag, overwrite the tag, and perform various other operationsdescribed herein.

Example Embodiments

Reference is first made to FIG. 1 . FIG. 1 shows a network domain orsystem 10 comprising a plurality of network devices 20(1)-20(N) that areall synchronized to a common time reference 30. That is, all of thenetwork devices 20(1)-20(N) of interest in the network domain 10 havethe same global time reference, determined by IEEE 1588 Precision TimeProtocol (PTP) or other methods now known or hereinafter developed. Forsimplicity, the term “node” is also used herein synonymously with theterm “network device”.

A packet 40 enters the network domain 10 at some edge node, e.g.,network device 20(1) in the example of FIG. 1 , and departs the networkdomain 10 at another edge node, e.g., network device 20(N). Presentedherein are techniques to determine latency at any point in the networkdomain for a packet as it traverses through the network domain 10. Oneparticular latency measure that is of interest is the end-to-endlatency, that is, the elapsed time (latency) for a packet to travelbetween an ingress port of edge network device 20(1) to an egress portof edge network device 20(N) of network domain 10.

A timestamp tag (TTAG) is inserted into a packet 40 by the edge networkdevice 20(1) of the network domain 10. The TTAG includes timestampinformation indicating time of arrival at network device 20(1). All ofthe network devices in the network domain 10 that receive the packet 40(with the inserted TTAG) can perform measurements based on the timestampinformation contained in TTAG inserted into packet 40, and perform otheroperations, including adding another TTAG, overwriting an existing TTAG,adding another timestamp value into an existing TTAG, etc., as willdescribed in more detail hereinafter. As indicated in FIG. 1 , anynetwork device in the network domain 10 can measure and report thelatency based on the TTAG contained in a packet. However, not allnetwork devices must understand a TTAG. In cases in which a networkdevice does not understand a TTAG contained in a packet, the TTAG can beskipped as part of packet processing or in the case of Ethernet packets,some switches will process the packet up to the TTAG and skip the restof the packet.

The network devices 20(1)-20(N) shown in FIG. 1 can be any networkdevice now known or hereinafter developed, including a switch, router,gateway, a software stack on a host device, virtual network interfacecards (VNICs) virtual switches, physical network interface cards(including those that support virtualization).

FIG. 1 further shows a network management station 50 that may take avariety of forms, e.g., server computer, virtualized server, etc., thatcommunicates with each network device 20(1)-20(N) for purposes ofconfiguring the network devices to insert TTAGs, make latencymeasurements, report latency measurements, and to receive latencymeasurements from the network devices 20(1)-20(N).

Turning now to FIG. 2 , a more detailed description is provided for thecomponents of a network device that are configured to perform the TTAGinsertion and latency measurement operations presented herein. FIG. 2shows a simplified diagram of two network devices 20(1) and 20(2),though it should be understood that each network device 20(1)-20(N) in anetwork domain that is to participate in the techniques presented hereinis configured in a similar manner as that shown for network devices20(1) and 20(2) in FIG. 2 . Specifically, each network device20(1)-20(N) includes multiple ports, and for simplicity an ingress port21 and egress 22 are shown in FIG. 2 . Furthermore, each network deviceincludes a timestamp logic unit 23, a latency measurement unit 24,packet processing logic 26, a central processing unit (CPU) 28 andmemory 29. The packet processing logic 26 is representative of theconventional packet processing components in a network device, such asbuffers, switch tables, switch fabric, queues, etc., that operate todetermine whether to drop, forward (and via a particular egress port),switch, etc., a particular packet based on the contents of the header ofthe packet. The details of the packet processing logic 26 are notdescribed herein because they are well known in the art, and do notpertain to the timestamping techniques presented herein.

The timestamp logic unit 23 generates a timestamp upon arrival of thepacket at an ingress port 21 of the network device. The timestamp iswith respect to the common time reference 30 used by all network devicesin the network domain. The timestamp logic unit 23 may insert the TTAGinto a packet 40 immediately upon arrival at the ingress port, and thenforward the packet to be processed by the packet processing logic 26,insert the TTAG in parallel with the processing of the packet by thepacket processing logic 26, or after processing of the packet by thepacket processing logic 26. Examples of various formats of a TTAG arepresented hereinafter in connection with FIGS. 3A-3D. The TTAG isinserted in any manner that does not interfere with the normalprocessing of the packet by the network devices.

The timestamp logic unit 23 may also be configured to insert additionalinformation into a TTAG, including one or more bits to indicate avalidity of the timestamp value, one or more bits to indicate a timingprecision of the timestamp value. In general, precision is system ornetwork domain wide and is pre-negotiated among the network devices withrespect to the common time reference 30. When a new timestamp value isto be inserted into a packet, a network device uses either ingress porttimestamp from the common time reference 30 (synchronized clock) or aninvalid value of zero. Invalid values are preserved across the networkdomain, as described further hereinafter.

Since any device can serve as an edge node in a network domain, eachnetwork device includes latency measurement 24 which is configured toperform a latency computation (current time minus the timestamp valuecontained in a TTAG of a received packet). For example, the latencymeasurement unit 24 in network device 20(2) may compute the latencyassociated with packet 40 using the timestamp value contained in theTTAG inserted by edge network device 20(1).

The CPU 28 may perform higher level latency analysis and reportingoperations based on software instructions contained in memory 29. Thememory 29 may also serve for additional storage of latency measurements.The CPU 28 may send latency measurements to a local or remotely locatedcomputing device that is used by a network administrator to monitorperformance of network domain 20. Moreover, the CPU 28 in any givennetwork device may receive commands or instructions from a networkmanagement station (FIG. 1 ) to control the TTAG-related operations in anetwork device, latency measurements made by a network device, etc.

Memory 29 may comprise read only memory (ROM), random access memory(RAM), magnetic disk storage media devices, optical storage mediadevices, flash memory devices, electrical, optical, or otherphysical/tangible memory storage devices. The CPU is, for example, amicroprocessor or microcontroller. Thus, in general, the memory 29 maycomprise one or more tangible (non-transitory) computer readable storagemedia (e.g., a memory device) encoded with software comprising computerexecutable instructions and when the software is executed (by the CPU28) it is operable to perform the operations described herein.

The timestamp logic unit 23 and latency measurement unit 24 may beembodied by digital logic gates configured to perform the operationsdescribed herein, or in another form, by software stored in memory 29and executed by CPU 28 to perform the operations described herein. Inanother example, the timestamp logic unit 23 and latency measurementunit 24 may be integrated or embedded with the packet processing logic26, which itself may be embodied by one or more application specificintegrated circuits (ASICs).

As shown in FIG. 2 , when a network device receives from another networkdevice a packet that includes a TTAG, there are several options foroperations that may be performed. First, the network device can measurelatency from the edge node or any other node that inserted a TTAG in thepacket. Second, the network device can do nothing, leave the TTAG as isand process the packet in the ordinary course. Third, the network devicecan overwrite an existing TTAG in the packet with a new TTAG (andtimestamp of arrival) at this network device. Fourth, the network devicecan insert an additional TTAG into the packet. For example, multipleTTAGs can be inserted such as through tunnels or if negotiated acrossports. Fifth, the network device can insert an additional timestampvalue (based on time of arrival at this network device) into an existingTTAG of the packet. The CPU 28 in one or more network devices may beconfigured, through software stored in memory 29, to insert additionalTTAGs into a packet, overwrite an existing TTAG or insert anothertimestamp value in a TTAG as described further hereinafter. In any case,the network device processes the packet as it normally would if the TTAGwere not present. As shown in FIG. 2 , network device 20(2) sends packet40 on in the network domain with any existing TTAGs, a newly overwrittenTTAG, etc., under control of the CPU 28.

Reference is now made to FIGS. 3A-3E for examples of various formats ofTTAGs. FIG. 3A illustrates a first basic form of a TTAG 100, including afirst Type subfield 110 and a second Timestamp subfield 120. The Typesubfield 110 is used to identify the “type” as a TTAG which allows anynetwork device to recognize the TTAG. The independence of timestampinformation contained in a TTAG from any other existing format liberatescurrent network devices or CPUs to determine system-wide time. In oneexample, the Type subfield 110 is 8 bytes such as that specified by anEthertype subfield in an Ethernet frame. The Timestamp subfield 120 is a48 bit number having a format of an unsigned rolling 48 bit binarynumber value, e.g., having 100 picosecond resolution. When clock timeincrements to zeros for all 48 LSBs, the Timestamp subfield uses a valueof one instead. A value of one repeats unlike one's complement. Thus, alower 48 bit clock time of 0 and 1 both map to Timestamp subfield valueof 1.

FIG. 3B shows an example format of a TTAG 101 with an explicit validitybit shown at 130. The validity bit 130 is configured so that if it takeson a first value, e.g., logic “1”, the timestamp value in Timestampsubfield 120 is valid, and if the validity bit takes on a second value,e.g., logic “0”, the timestamp value in the Timestamp subfield 120 isinvalid. Invalid timestamp values are preserved across the networkdomain by other network devices that receive a packet with a TTAGindicated to contain an invalid timestamp value.

There is another way to signify an invalid timestamp value in a packetwithout using the explicit validity bit 130. A Timestamp subfield valueof zero represents an invalid timestamp. Thus, when the value containedin Timestamp subfield 130 is all zeros, a network device construes thisas indicating that the timestamp contained in the TTAG is invalid. Thesubfield can be compatible with timestamp always valid in the networkwhen invalid capability is disabled in the network domain. Thus, apredetermined bit pattern (e.g., all zeros) in the Timestamp subfield130 indicates that the timestamp information of the Timestamp subfieldis not valid.

FIG. 3C illustrates another format of a TTAG shown at reference numeral102. In this example, there is an additional precision subfield 140 thatcontains a bit pattern configured to indicate precision of the timestampvalue contained in the Timestamp subfield 120. The concept ofnetwork-wide pre-negotiated precision was described above.

FIG. 3D illustrates still another format of a TTAG shown at referencenumeral 103. This example shows that there is both the explicit validitybit 130 and precision subfield 140.

FIG. 3E illustrates yet another format of a TTAG shown at referencenumeral 104. TTAG 104 includes multiple Timestamp fields 120(1), 120(2),etc. Each Timestamp subfield can contain a different timestamp valueinserted by the same network device or by different network devices.

Turning to FIG. 4 , a general diagram is shown of a packet 40 having oneor more TTAGs 100(1), 100(2), etc., therein. In the simple case, apacket will have only one TTAG at any given time. However, there isutility in the capability of multiple TTAGs in a packet. For example,multiple TTAGs can be inserted in situations when packets areencapsulated in tunnels or if negotiated across ports of networkdevices. Furthermore, each packet that has a TTAG inserted into it doesnot affect the networking operations performed by any network devicethat receives the packet. Any network device can obtain information fromthe TTAGs contained in packets and thereby obtain visibility to latencywithin the network.

In some implementations of the techniques described herein, the numberof TTAGs that can be inserted into a packet is limited in number to, forexample, six (6) or some number between one (1) and ten (10). In otherimplementations, the number of TTAGs that can be inserted into a packetis unlimited, in which any device that receives the packet within thenetwork for passing the packet to a destination from a source can inserta TTAG into the packet. In some implementations, when the maximum numberof TTAGs that can be inserted into a packet is reached, downstreamnetwork devices cannot insert TTAGs into the packet. In yet otherimplementations, when the maximum number of TTAGs that can be insertedinto a packet is reached, downstream network devices are allowed toover-write TTAGs on a first-in, first-out basis.

There are numerous possibilities for locating the TTAG information inthe packet. The TTAG can be inserted within a Layer 2 portion of thepacket. This is in contrast to conventional approaches that performapplication-specific packet time measurements at Layer 3. For example,one conventional packet time measurement approach collects runtimemeasurement of packets based on an application-specific determination ofpacket arrivals at Layer 3, as opposed to incorporating timestamp taginformation directly into all packets at Layer 2 as accomplished usingthe techniques presented herein.

In some implementations, such as for Internet Protocol Version 4 (IPv4)or IPv6 packets, the TTAG can be provided immediately after the virtuallocal area network (VLAN) subfield and immediately before the IPv4 orIPv6 field in the packet header portion of the packet, in which the TTAGis meshed in the protocol stack within the header portion of the packet.Other locations for insertions of the TTAG within a packet may beenvisioned while remaining within the spirit and scope of the techniquespresented herein.

Turning now to FIG. 5 , a flow chart is presented that illustrates anoperational flow 200 with respect to network devices that insert andinterpret TTAGs in packets as the packets traverse through a networkdomain. At 210, a packet at a first network device (e.g., ingress edgenode for the packet) of a network domain is received. The packet is fortransport through the network domain, and the network domain includes aplurality of network devices, e.g., as depicted in FIG. 1 . At 220, thefirst network device generates timestamp information indicating time ofarrival of the packet at the first network device. As explained above inconnection with FIG. 1 , the timestamp is generated with respect to atime reference that is common across all of the network devices in thenetwork domain. At 230, the first network device inserts into the packeta tag that comprises at least a first subfield and a second subfield.The first subfield comprising a type indicator to signify to othernetwork devices in the network domain that the tag includes timestampinformation, and the second subfield includes or contains the timestampinformation.

At 240, the first network device sends the packet to another networkdevice in the network domain, using the normal packet processingfunctions for the packet. At 250, another network device in the networkdomain receives the packet, and can perform any one or more of: (i)measuring latency with respect to first network device based ontimestamp information in tag, (ii) overwriting tag with new tag, (iii)adding an additional tag to the packet, and (iv) adding an additionaltimestamp to an existing tag, or (v) doing nothing and processing thepacket in the normal course without performing any of operations(i)-)(iv).

At 250, the network device sends the packet on in the network in theordinary course of packet processing. Operations 240 and 250 arerepeated at subsequent network devices in the network domain as thepacket travels through the network domain.

As explained above in connection with FIGS. 1 and 2 , a networkmanagement station may receive reports as to latency values measured bynetwork devices in the network domain. The network management stationmay also configure the various network devices to perform morespecialized tagging of packets, depending on certain applicationssupported in the network, tunnels supported in the network, etc. Thelatency measurements made by network devices at the edge of the networkdomain and at various points in between allow a network administrator tounderstand how the network domain is handling traffic and whether thereare network congestion issues within a particular portion of the networkdomain.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking andparallel processing may be advantageous.

The above description is intended by way of example only.

What is claimed is:
 1. A method comprising: receiving a network trafficpacket at a first network device for transport within a network domainthat comprises a plurality of network devices, the network trafficpacket comprising a header with a Layer 2 portion and a Layer 3 portion,wherein the plurality of network devices have a common time referenceand the network traffic packet does not include time information sourcedfrom the common time reference, and wherein the first network device isat an ingress edge of the network domain for the network traffic packet,wherein the network traffic packet is received at the first networkdevice from outside of the network domain; generating, by the firstnetwork device, timestamp information indicating time of arrival of thenetwork traffic packet within the network domain; inserting into theLayer 2 portion of the header of the network traffic packet a first tagthat is distinct from contents of the network traffic packet, whereinthe first tag comprises at least a first type subfield and a multiplesecond subfield subfields, wherein the first type subfield comprising atype an indicator to signify to other network devices in the networkdomain that the first tag includes timestamp information data, andwherein at least one of the multiple second subfield subfields includesthe timestamp information indicating time of arrival of the networktraffic packet with an associated precision, wherein the timestampinformation is configured to be used by each device in the networkdomain that receives the network traffic packet as an indication of whenthe network traffic packet entered the network domain; and sending thenetwork traffic packet from the first network device to another networkdevice in the network domain.
 2. The method of claim 1, furthercomprising: receiving at a second network device the network trafficpacket sent from the first network device; determining a time of arrivalof the network traffic packet at the second network device; andmeasuring latency of the network traffic packet within the networkdomain based on the time of arrival of the network traffic packet andthe timestamp information contained in the first tag of the networktraffic packet.
 3. The method of claim 1, further comprising: at each ofthe other network devices in the network domain: receiving the networktraffic packet sent by the first network device; determining time ofarrival of the network traffic packet; and measuring latency of thenetwork traffic packet within the network domain based on the time ofarrival of the network traffic packet and the timestamp informationcontained in the first tag of the network traffic packet.
 4. The methodof claim 1, wherein the first type subfield of the first tag is anEthertype subfield.
 5. The method of claim 1, further comprisinginserting in the first tag a validity bit that indicates whether or notthe timestamp information is valid.
 6. The method of claim 1, whereininserting into the network traffic packet comprises inserting apredetermined bit pattern in the second subfield to indicate that thetimestamp information of the second subfield is not valid.
 7. The methodof claim 1, further comprising: receiving at a second network device thenetwork traffic packet sent from the first network device; and insertingan additional a second tag into the network traffic packet, theadditional second tag including timestamp information representing timeof arrival of the network traffic packet at the second network device,wherein the second tag is distinct from contents of the network trafficpacket and distinct from the first tag.
 8. A method comprising:receiving a network traffic packet at an egress port of a first networkdevice, the network traffic packet not including time informationsourced from a common time reference and having been transported withina network domain that comprises a plurality of network devices, thenetwork traffic packet comprising a header with a Layer 2 portion and aLayer 3 portion, wherein the plurality of network devices have a commontime reference, and wherein the first network device is at an egressedge of the network domain for the network traffic packet; determining atime of arrival of the network traffic packet at the first networkdevice; extracting, by the first network device, from the Layer 2portion of the header of the network traffic packet a tag that isdistinct from contents of the network traffic packet, the tag includinga type subfield and multiple timestamp subfields, wherein the typesubfield comprises an indicator to signify to other network devices inthe network domain that the tag includes timestamp information, whereinat least one of the multiple timestamp subfields including timestampinformation indicating time of arrival of the network traffic packetwithin the network domain with an associated precision, wherein thenetwork traffic packet was received within the network domain at asecond network device, wherein the second network device is arranged atan ingress edge of the network domain for the network traffic packet;and determining the a latency of the network traffic packet within thenetwork domain by calculating an elapsed time between the timestampinformation and the arrival of the network traffic packet at the egressport of the first network device.
 9. The method of claim 8, whereinextracting the timestamp information comprises extracting at least afirst subfield and a multiple second subfield subfields, the firstsubfield being the type subfield comprising a type the indicatorsignifying that the multiple second subfield includes subfields includethe timestamp information.
 10. The method of claim 9, wherein extractingthe first subfield comprises extracting an Ethertype subfield.
 11. Themethod of claim 9, wherein extracting the timestamp informationcomprises extracting a predetermined bit pattern in the second subfieldto indicate that the timestamp information of the second subfield is notvalid.
 12. The method of claim 8, wherein extracting the timestampinformation comprises extracting timestamp information for each of theplurality of network devices that the network traffic packet traversedtravelling from the second network device to the first network device.13. The method of claim 8, wherein extracting the timestamp informationcomprises extracting a validity bit that indicates whether or not thetimestamp information is valid.
 14. The method of claim 8, whereinextracting the timestamp information comprises extracting the timestampinformation from an Open Systems Interconnection model Layer 2 portionof the packet.
 15. An apparatus comprising: a plurality of ports eachconfigured to receive and send network traffic packets in a networkdomain, the network traffic packets comprising a header with a Layer 2portion and a Layer 3 portion, wherein at least one of the plurality ofports is an egress port at an egress edge of the network domain, andwherein the network traffic packets do not include time informationsourced from a common time reference; a timestamp logic unit configuredto extract, from the Layer 2 portion of the header of the networktraffic packet a tag that is distinct from contents of the networktraffic packet, the tag including a type subfield and multiple timestampsubfields, wherein the type subfield comprises an indicator to signifyto other network devices in the network domain that the tag includestimestamp information, wherein at least one of the multiple timestampsubfields including timestamp information indicating time of arrival ofthe a network traffic packet at the network domain with an associatedprecision, wherein the timestamp information is generated at a networkdevice arranged at an ingress edge of the network domain for the networktraffic packet; and a latency measurement unit configured to determine atime of arrival of the network traffic packet at the egress port anddetermine an elapsed time between the arrival of the network trafficpacket at the network domain and the arrival of the network trafficpacket at the egress port.
 16. The apparatus of claim 15, wherein thetimestamp logic unit is configured to extract at least a first subfieldand a multiple second subfield subfields, the first subfield being thetype subfield comprising a type the indicator signifying that themultiple second subfield includes subfields include the timestampinformation.
 17. The apparatus of claim 16, wherein the timestamp logicunit is configured to extract the first subfield as an Ethertypesubfield.
 18. The apparatus of claim 16, wherein the timestamp logicunit is configured to extract the timestamp information by extracting apredetermined bit pattern in the second subfield to indicate that thetimestamp information of the second subfield is not valid.
 19. Theapparatus of claim 15, wherein the latency measurement unit isconfigured to determine the elapsed time from the time of arrival of thenetwork traffic packet at the egress port and the timestamp information.20. The apparatus of claim 15, wherein the timestamp logic unit isconfigured to timestamp information from an Open Systems Interconnectionmodel Layer 2 portion of the packet.
 21. The apparatus of claim 15,wherein the timestamp logic unit is configured to extract the timestampinformation by extracting a validity bit that indicates whether or notthe timestamp information is valid.
 22. The apparatus of claim 15,wherein the timestamp logic unit is configured to extract the timestampinformation by extracting the timestamp information from an Open SystemsInterconnection model Layer 2 portion of the packet.
 23. The method ofclaim 1, wherein generating the timestamp information indicating time ofarrival of the network traffic packet within the network domaincomprises determining a time of arrival at the first network device. 24.The method of claim 1, wherein inserting into the packet the tagcomprises inserting the tag in an Open Systems Interconnection modelLayer 2 portion of the packet.
 25. A method, comprising: capturing, at afirst network device that is configured to forward network trafficpackets, a network traffic packet for transport within a network domainthat comprises a plurality of network devices, the network trafficpacket comprising a header with a Layer 2 portion and a Layer 3 portion,wherein the first network device is at an ingress edge of the networkdomain for the network traffic packet, and wherein the network trafficpacket does not include time information sourced from a common timereference and is transported to the first network device from outside ofthe network domain; generating, by the first network device, timestampinformation indicating time of capture of the network traffic packet atthe first network device; including in the Layer 2 portion of the headerof the network traffic packet a tag that is distinct from contents ofthe network traffic packet, wherein the tag comprises at least a firstsubfield and multiple second subfields, wherein the first subfieldcomprising a type subfield comprises an indicator to signify to othernetwork devices in the network domain that the tag includes timestampinformation, and wherein at least one of the multiple second subfieldscomprises a timestamp value that indicates time of capture of thenetwork traffic packet at the first network device with an associatedprecision; and sending the network traffic packet from the first networkdevice to a second network device.
 26. The method of claim 25, whereinthe tag further includes a subfield indicating a validity of the networktraffic packet.
 27. The method of claim 25, wherein the network trafficpacket containing the tag is formatted such that the network trafficpacket can be processed by a second network device that is notconfigured to understand the tag.
 28. The method of claim 25, furthercomprising: capturing the network traffic packet at the second networkdevice; determining time of capture of the network traffic packet at thesecond network device; and measuring latency of the network trafficpacket within the network domain based on the time of capture of thenetwork traffic packet at the second network device and the timestampinformation included in the tag of the network traffic packet.
 29. Themethod of claim 25, wherein the tag further includes a precisionsubfield indicating precision of the timestamp value in an associatedsecond subfield.
 30. An apparatus comprising: an ingress port configuredto receive a network traffic packet, the network traffic packetcomprising a header with a Layer 2 portion and a Layer 3 portion,wherein the network traffic packet does not include time informationsourced from a common time reference; and a timestamp logic unitconfigured to generate timestamp information based on time of arrival ofthe network traffic packet at the ingress port and insert into the Layer2 portion of the header of the network traffic packet, a tag that isdistinct from contents of the network traffic packet, wherein the tagcomprises subfields including a type subfield and multiple timestampsubfields, wherein the type subfield comprises an indicator to signifyto other network devices that the tag includes timestamp information,and wherein at least one of the multiple timestamp subfields having anassociated precision subfield to indicate a precision for a timestampvalue for time of arrival of the network traffic packet in an associatedtimestamp subfield.
 31. The apparatus of claim 30, wherein the networktraffic packet further comprises a validity check subfield.
 32. Theapparatus of claim 30, further comprising: a latency measurement unitconfigured to determine latency of the network traffic packet traversinga network domain from a first ingress port of the network domain to asecond port based on at least one of the subfields containing timestampinformation.
 33. The apparatus of claim 30, wherein the type subfieldcomprises an Ethertype subfield.
 34. An apparatus comprising: aplurality of ports, each port configured to receive and send networktraffic packets in a network domain, the network traffic packetscomprising a header with a Layer 2 portion and a Layer 3 portion,wherein at least one of the plurality of ports is an ingress port at anedge of the network domain, wherein the network traffic packets do notinclude time information sourced from a common time reference; atimestamp logic unit configured to insert into the Layer 2 portion of areceived network traffic packet a first timestamp information indicatingtime of arrival of the received network traffic packet at the networkdomain from outside the network domain, wherein the first timestampinformation is generated with reference to the ingress port at the edgeof the network domain to enable latency measurements based on the firsttimestamp information, and the first timestamp information is distinctfrom contents of the received network traffic packet, the firsttimestamp information having an associated precision; and wherein theLayer 2 portion of the received network traffic packet further includesa type subfield that comprises an indicator to signify to other networkdevices received network traffic packet includes timestamp information.35. The apparatus of claim 34, wherein the type subfield is an Ethertypesubfield.
 36. A system comprising the apparatus of claim 34, and furthercomprising another apparatus that receives the network traffic packetthat includes the first timestamp information.
 37. The method of claim1, wherein inserting comprises inserting a plurality of additional tagsinto the network traffic packet, each additional tag including differenttimestamp information, wherein each additional tag is distinct from thefirst tag and distinct from other additional tags.
 38. The method ofclaim 8, wherein the network traffic packet includes multiple tags, eachtag of the multiple tags including timestamp information indicating timeof arrival of the network traffic packet at a different network device,and wherein each tag of the multiple tags is distinct from other tags.39. The method of claim 38, wherein one or more tags of the multipletags have been overwritten by a network device when the network trafficpacket has a maximum number of tags.